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ABSTRACT 



Smart Start is a multi-disciplinary, comprehensive, 
community-based initiative to serve North Carolina children under age 6 and 
their families to ensure that all children enter school healthy and prepared 
to succeed. To achieve this, local county partnerships have focused both 
their attention and their funds on three major areas of service 
implementation: child care, family programs, and health services. The 
evaluation data collection strategies have included measuring changes in 
child care quality, family functioning, and children's receipt of health 
services, as well as the long-term outcome of school success. The most 
notable recent finding is the significantly higher quality of child care as 
observed in randomly selected preschool classrooms in the pioneer Smart Start 
counties in 1996-97 compared to 1994-95. This report details the four main 
goals of Smart Start: (1) children are healthy and prepared to succeed in 

school; (2) families effectively fulfill their role as primary providers, 
nurturers, and teachers,* (3) high quality, affordable services for children 
will be available; and (4) North Carolina counties value children and 
families by providing options and resources, and encouraging collaboration. 
Related to each goal, the report presents the main data collection strategies 
being used to evaluate progress towards the goal, followed by brief summaries 
of recent results and key findings from earlier reports. (JPB) 
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NORTH CAROLINA’S SMART START INITIATIVE: 
1996-97 ANNUAL EVALUATION REPORT 



Smart Start is a multi-disciplinary, comprehensive, community-based initiative to serve North 
j Carolina children under age 6 and their families. The major long-range goal of Smart Start is to 

ensure that all children enter school healthy and prepared to succeed. To achieve this goal, local 
county partnerships have focused both their attention and their funds on three major areas of 
service implementation: child care, family programs, and health services. The evaluation team 
considers changes in these areas to be intermediate-term outcomes, that is, changes that should 
lead to the longer-range goal of increased preparedness for school. Therefore, the evaluation data 
collection strategies have included measuring changes in child care quality, family functioning, 
and children’s receipt of health services, as well as the long-term outcome of school success. 

The most notable recent finding is the significantly higher quality of child care as observed 
in randomly selected preschool classrooms in the pioneer Smart Start counties in 1996-97 
compared to 1994-95. These results are further described on pages 4-5 and in a separate, 
detailed report by the evaluation team, Effects of Smart Start on Preschool Child Care Quality. 
We highlight the child care results here because they are an indication that one of Smart Start’s 
main strategies for improving children’s preparedness for school — enhancing the quality of their 
child care environment — seems to be having a positive effect. 

This report is organized by the four main goals of Smart Start. Related to each goal, the report 
presents the main data collection strategies being used to evaluate progress towards the goal 
followed by brief summaries of recent results and key findings from earlier reports. Many of 
these findings are in separate and more detailed reports, listed at the end of this document. 



Goal 1. Children are healthy and prepared to succeed in school. 

North Carolina public schools obtain health and readiness data on entering kindergartners in a 
variety of different ways, with no uniform database available to monitor children’s progress 
across counties. Thus, the evaluation team has developed a standard way of collecting these data 
in selected counties. Randomly selected teachers from all 44 Smart Start counties rated 3900 
randomly sampled children on the Kindergarten Teacher Checklist in 1995 to measure children’s 
cognitive, language, motor, and social skills. Data collectors coded immunization, screening, 
and other health data from the Kindergarten Health Assessment forms of 9700 randomly selected 
kindergartners in 20 Smart Start counties, also in 1995. These data from kindergarten teachers 
and children’s health records serve as baseline data. Similar data will be gathered in the fall of 
1 997 to determine whether kindergartners’ skills and health status have changed over the past 2 
years. Several findings about health and preparedness for school are listed below. 
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• In general, NC children enter kindergarten with a wide range of skills, however 18% of 
kindergartners in Smart Start counties were judged by their teachers as not ready to 
participate successfully in school. 

• Kindergartners who had attended child care had more skills than those who did not attend 
child care, which is consistent with other research demonstrating that child care can promote 
cognitive, language, and social skills. 

• Children from poor families had fewer skills than those from non-poor families, supporting 
long-standing research that poverty is a component of school failure. These findings also 
support Smart Start’s focus on getting more children from poor families into child care and 
improving the overall quality of child care. 

• Kindergartners from partnerships spending more money on child care had significantly more 
skills than kindergartners from partnerships spending less money on child care. Because 
these results are correlational (not causal), we must cautiously note that counties with more 
children who are better prepared for school may also be counties that have a history of more 
funding for early childhood programs. 

• Kindergartners from poor families in counties that received more of their full-funding 
allocation were rated as having significantly more skills than kindergartners from poor 
families in counties that received proportionately less of their assessed full-funding 
allocation. The counties that received less of their full-funding allocation tended to be more 
urban so this finding likely indicates that poor children in urban areas are generally doing less 
well than poor children in rural areas. 

• Most kindergartners were fully immunized once they entered kindergarten, but only 53.3% 
had been immunized on time. Many children apparently are given their last immunizations 
right before school starts, but these children should have received the immunizations earlier. 

• Of entering kindergartners, 2% failed a hearing test, 7% failed a vision test, and 25% had at 
least one identified health problem. These are the kinds of health needs that Smart Start 
programs are attempting to find and treat earlier than kindergarten entry. The evaluation 
team will monitor these types of health needs over time to note possible changes. 

• Speech problems seem the most likely developmental problem to receive treatment in the 
preschool years, with over 40% of entering kindergartners who were noted to have a speech 
problem having already been referred to a DEC. Child care centers, health providers, and 
parents are more likely to perceive the need for speech intervention compared to other 
problems, so more children with speech difficulties have received treatment before they enter 
school. Even so, almost 60% of children with speech problems had not yet been referred. 

The evaluation team will again assess these areas of kindergarten health and skills in the fall of 

1997, using the 1995 data as a comparison point.' Entering kindergartners and their families in 
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the 12 pioneer partnerships will have been “exposed” to 4 years of Smart Start efforts and more 
counties have entered into Smart Start since the first round of data collection. This will provide 
more opportunities for comparisons to better understand possible Smart Start effects on 
children’s health and preparedness for school. 



Goal 2. Families effectively fulfill their role as primary providers, nurturers and teachers. 

To gather information about families served by Smart Start, the evaluators interviewed 356 
families from the pioneer partnerships who had received some type of child care, health, or 
family support service. In each county, the sample was selected in proportion to the county’s 
distribution of Smart Start funds in the three main areas. For example, in a partnership that spent 
most of their funds on child care, most of the families interviewed were randomly selected from 
child care lists. Questions on the interview covered child care, health, and community services. 
Some questions about families’ participation in educationally important activities were 
previously asked in a national sample of families in the 1993 National Household Education 
Survey and other questions about family strengths were previously asked of 700 other NC 
families by the Family Council of North Carolina. Smart Start families can be compared to the 
other samples in these domains. The findings from these interviews will be detailed in a report 
to be completed in June, 1997, but some important facts are noted below. 

• Smart Start serves a wide range of North Carolina families, from the unemployed to the 
working poor to the well-off. Sixty-three percent (63%) of low-income and 90% of middle- 
income Smart Start families in our sample are employed. 

• Families participating in Smart Start engage in educationally important activities with their 
child (such as reading, playing number games, telling stories, singing together) as often, if 
not more often, than a national sample of parents of preschoolers. The similarity between the 
Smart Start and national sample is even more striking because of the larger proportion of 
low-income parents included in the Smart Start sample. 

• A high percentage of Smart Start families (79%) report that they have the strength to cope 
with the pressures on them, although this percentage is slightly lower than found in a random 
sample of NC families (84%). 

• Participation in religious activities was high among Smart Start families (60%) and for many 
families was the only involvement in community groups reported; 25% of Smart Start 
families were not involved in any community group. 

• Parents learn about services for their children and themselves through word of mouth, friends 
and relatives, and the phone book. An activity frequently funded by Smart Start — creating a 
directory of community services — was ranked fourth. These findings indicate that there is a 
continuing need for distribution of information about child and family services to a wide 
variety of agencies and people, not just to professionals or through service agencies. 
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Especially given the high percentage of families involved with church or religious 
organizations, providing information through these groups is a way to reach a large number 
of people. 

• Almost one-fourth of families interviewed said they needed parenting education programs. 

• Low-income families were more than twice as likely to report one or more barriers to 
obtaining needed services, including cost of services, inconvenient hours, lack of child care, 
and negative attitudes of the staff at service agencies. 

Smart Start is funding a wide variety of family-focused programs. Most of them appear to be 
desired by a large number of families, and many are serving the need for information, education, 
and training as expressed by many parents. It should be noted that families know programs by 
names, not funding sources. Many families whom we knew were receiving Smart Start-funded 
services did not know that they were a part of Smart Start. Therefore, further evaluation of 
family progress will be difficult to link directly to Smart Start participation if we must rely on 
only parental self-report. The evaluation team is currently working with a few partnerships to 
conduct longitudinal studies of small groups of children and families who are participating in 
various Smart Start services. This strategy may provide better ways of evaluating families over 
the course of their participation in Smart Start programs. 



Goal 3. High quality, affordable services for children and families will be available. 

Data on the services and quality of child care provided in 1 80 child care centers across the 
pioneer partnership were gathered through preschool classroom observations and director 
interviews in 1994-95 and again in 187 centers in 1996-97. The Early Childhood Environment 
Rating Scale (ECERS) was used as the rating of developmental appropriateness in randomly 
selected preschool classrooms in these centers. Highlights of these very recent findings are 
presented below. (For more detail, see Effects of Smart Start on Preschool Child Care Quality.) 

• In pioneer Smart Start partnerships, the quality of child care was significantly greater in 1 996 
than in 1994, both as measured by the percentage of AA centers and by observations of 
quality in preschool classrooms. From 1994 to 1996 the percentage of classes rated as 
“good” or above on the ECERS increased from 14% to 25%. 

• These quality changes appear to be related to Smart Start participation as evidenced by four 
specific findings. (1) The proportion of funds a partnership spent on child care quality 
improvements significantly predicted quality, with more funds spent on child care quality 
improvement positively related to ECERS quality. (2) The percentage of full-funding 
received by a partnership affected the influence of proportion of funds spent on child care. 
Proportion spent on child care was not as strongly associated with quality in counties that 
received a low percentage of their full-funding amount as it was in counties that received a 
high percentage of their full-funding amount. (3) The number of child care quality 
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improvement activities that an individual center participated in was significantly related to 
their ECERS quality score -- more activities were associated with better quality. (4) The 
average quality of the 91 centers that were visited in both 1994 and 1996 increased 
significantly, including an increase in the proportion of centers licensed at the higher AA 
level. These findings all support the conclusion that improved child care quality is associated 
with Smart Start participation. 

• The percentage of AA-licensed child care centers (a higher level of quality care than the A 
license) is increasing in many counties in NC, but it is increasing at a faster rate in Smart 
Start counties than in non-Smart Start counties. 

• The quality of care in centers that included children with disabilities was higher than that 
found in programs enrolling only nondisabled children, suggesting that high quality child 
care is available for children with disabilities and their families. 

• In 1994, 62% of the child care teachers in our sample from the pioneer counties had a NC 
Child Care Credential or an Associate or BA degree. In the 1996 sample, 74% had a NC 
Child Care Credential or higher degree. In the 1994 sample, teacher turnover was 32% 
compared to 10% in public schools and the average wage for a lead teacher was $5. 77/hour. 
The evaluation team is currently analyzing the recently gathered child care data to determine 
if Smart Start efforts have improved these and other variables related to the quality of care. 

• The median monthly child care fee for parents of preschoolers in 1994-95 was $240, ranging 
from $138 to $550 across the first Smart Start pioneer counties. In 1996, the median fee was 
$260, with a range from $120 to $622. More than half of all families interviewed expressed 
a need for financial assistance for child care and most were receiving it. Half of low-income 
families spent more than 10% of their income on child care. 

In summary, the most significant finding in the area of child care has been the overall increase in 
preschool classroom quality from 1994 to 1996. The difference is significant at a level that 
scientists term “moderate,” that is, a change that is meaningful and worthwhile. Finding a 
difference of this magnitude is particularly notable since, in Smart Start’s first years, it has been a 
diverse set of “treatments” implemented in a wide variety of settings with a varying degree of 
intensity. However, the mean level of quality even in 1996 was below that which one would 
hope to achieve eventually, so there is still room for improvement. The evaluation team is 
currently conducting observations in 100 child care programs in the round 3 and 4 partnerships. 
These will serve as baseline child care quality data in counties that have just begun or are still 
planning their Smart Start efforts. 
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Goal 4. NC counties value children and families by providing options and resources, and 
encouraging collaboration. 

Data about collaboration and community involvement within partnerships were gathered through 
qualitative studies using interviews, focus groups, and document reviews with various 
partnership members. Information about partnerships’ inclusion of children with disabilities was 
gathered by reviewing documents. Because the process of implementing Smart Start is so unique 
and has been noted at many national meetings as a model of a new approach to community 
initiatives, the evaluation team has each year included a qualitative research component to 
investigate the process of board and agency collaboration. Some of the key findings are 
summarized below. 

• All pioneer partnerships included children with disabilities and their families in their plans 
and allocated up to 10% of their funds for programs serving children with disabilities and 
their families. Inclusion is a strongly supported belief among partnership board members and 
service providers. (For more details, see Effects of Smart Start on Young Children with 
Disabilities and their Families.) 

• Across partnerships, agencies have increased their collaboration and cooperation in 
developing needed resources for young children and their families as noted by key 
participants. Many participants feel that this collaboration is one of the most striking 
successes of the initiative. (For more details, see Keeping the Vision in Front of You: 

Results from Smart Start Key Participant Interviews.) 

• Local partnership board members fear that politicization of Smart Start at the state level has 
created a defensive mentality among partnership boards that somewhat stifles innovation. 
Local board members believe that over the years there has been a retraction of local 
autonomy and increased pressure for more standardization and regulation of local 
partnerships. (For more details, see Reinventing Government? Perspectives on the Smart 
Start Implementation Process.) 

• Parent and business involvement in the local partnerships is essential for partnerships to fully 
involve their communities, but this is a continuing challenge for partnerships. Executive 
directors have assumed the major responsibility for supporting parent and business 
involvement. Partnerships that have a higher parent and business involvement rely on 
multiple strategies, including recruiting key community leaders, offering a wide range of 
roles to participants, and supporting participation. (For more details, see Bringing the 
Community Into the Process: Issues and Promising Practices for Involving Parents and 
Business in Local Smart Start Partnerships.) 

Through interviews and focus groups, the evaluation team has noted increased collaboration 
among individuals and agencies involved with young children and families. The coming 
together has not always been smooth and easy, but the long-term goal of improved child and 
family well-being has encouraged many already-motivated individuals in 44 counties in North 
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Carolina to share their ideas, information, and energies towards creating more and better 
programs in their own community. 

Future Work 

The findings summarized in this report are measurable, interim effects of Smart Start, that is, 
they are steps along the way to improved health and preparedness of children entering 
kindergarten. This ultimate goal is one that may be accomplished if the Smart Start efforts 
achieve their intermediate goals — better child care, improved well-being of families, and greater 
health resources for children. We noted in this report that child care quality is indeed improved 
as a result of Smart Start. Partnerships are spending 48% of their 1996-97 funds on child care, so 
we will continue to measure child care quality. We will also continue to measure family well- 
being and health care utilization, and in 1997-98 will undertake another major data collection 
effort on the health and school success outcomes for children. 



For more detailed information about the Smart Start Evaluation, 
please contact Dr. Donna Bryant at (919) 966-4295. 
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Appendix: Reports From the UNC Smart Start Evaluation Team 



Emerging Themes and Lessons Learned: The First Year of Smart Start (August, 1994) 

This report describes the first-year planning process of the pioneer partnerships and makes some 
recommendations for improving the process. 

Smart Start Evaluation Plan (September, 1994) 

This report describes our comprehensive evaluation plan, designed to capture the breadth of 
programs implemented across the Smart Start partnerships and the extent of possible changes 
that might result from Smart Start efforts. 

Keeping the Vision in Front of You: Results from Smart Start Key Participant Interviews 
(May, 1995) 

This report documents the process as pioneer partnerships completed their planning year and 
moved into implementation. 

North Carolina’s Smart Start Initiative: 1994-95 Annual Evaluation Report (June, 1995) 

This report summarizes the evaluation findings to date from both quantitative and qualitative 
data sources. 

Reinventing Government? Perspectives on the Smart Start Implementation Process 
(November, 1995) 

This report documents pioneer partnership members’ perspectives on 2 major process goals of 
Smart Start: non-bureaucratic decision making and broad-based participation. 

Center-based Child Care in the Pioneer Smart Start Partnerships of North Carolina (May, 
1996) 

This brief report summarizes the key findings from the 1994-95 data on child care quality. 

Effects of Smart Start on Young Children with Disabilities and their Families (December, 
1996) 

This report summarizes a study of the impact of Smart Start on children with disabilities. 

Bringing the Community into the Process: Issues and Promising Practices for Involving 
Parents and Business in Local Smart Start Partnerships (April, 1997) 

This report describes findings from interviews and case studies about the involvement of parents 
and business leaders in the Smart Start decision-making process. 

The Effects of Smart Start on the Quality of Child Care (April, 1997) 

This report presents the results of a 2-year study of the quality of child care in the 12 pioneer 
partnerships. 
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